METHOD AND APPARATUS FOR EYE GAZING SMART DISPLAY 


Background of the Invention 

1 . Technical Field 

The present invention relates to a method and apparatus 
for enhancing the perceived video quality on a display. 

2. Related Art 

Eye tracking devices generally are based upon the 
principal that the direction of a person's gaze is directly 
related to the relative position of a pupil of an eye of a 
person. These eye tracking devices may include processing 
capabilities that operate on a video image of the eye to 
determine the gaze direction of the eye. These image 
processing capabilities are enhanced by using a bright eye 
affect. The bright eye affect is a result of the highly 
reflective nature of a retina of the eye. The characteristic 
of the retina causes that a significant amount of a light 
that enters an eye is reflected back through the pupil. When 
light shines into an eye along the axis of a camera lens, the 
retina reflects a significant portion of the light back to 
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the camera. The pupil appears as a bright disk to the 
camera. This effect allows the pupil to be readily imaged 
and tracked. Eye tracking devices have been used for weapon 
control, market research, and as enablement for the disabled • 

Summary of the Invention 

The present invention provides a display system for 
providing a high resolution area in a region where a user is 
looking at a display screen of a display device. 

The present invention provides an apparatus comprising: 

an eye tracking system for determining an eye-gaze 
direction line of a user looking at a display screen of a 
display device; 

an eye-gaze tracking module for extracting the eye-gaze 
direction from the eye tracking system and for determining an 
intersection point where the eye-gaze direction line 
intersects with the display screen; 

wherein the eye-gaze tracking module sends the 
intersection point data to a scalable video decoder; and 

wherein the scalable video decoder receives an encoded 
video stream and provides a first set of higher video 
resolution data to a first region surrounding the 
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intersection point on the display screen and a second set of 
lower video resolution data to a second region of the video 
screen. 

Another embodiment of the apparatus of the present 
invention comprises: 

an eye tracking system for determining an eye-gaze 
direction line of a user looking at a display screen of a 
display device; 

an eye-gaze tracking module for extracting the eye-gaze 
direction from the eye tracking system and for determining an 
intersection point where the eye-gaze direction line 
intersects with the display screen; 

wherein the eye-gaze tracking module sends the 
intersection point data to a scalable video encoder; and 

wherein the scalable video encoder receives a source 
video stream and provides an encoded first set of higher 
video resolution data to a remote video decoder and an 
encoded second set of lower video resolution data is provided 
to the remote video decoder. 

The present invention provides a method comprising: 

providing a display screen; 

determining an eye-gaze direction line of a user looking 
at the display screen; 
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determining an intersection point where the eye-gaze 
direction line intersects with the display screens- 
providing a video stream to a scalable video device; and 
wherein the scalable video device provides a first set 
of higher video resolution data to a first region surrounding 
the intersection point on the display screen and a second set 
of lower video resolution data to a second region of the 
video screen. 

Brief Description of the Drawings 

For an understanding of the present invention, reference 
should be made to the following detailed description taken in 
connection with the accompanying drawings wherein: 

FIG. 1 illustrates a schematic view of a display system; 

FIG. 2 illustrates a plan view of a display screen 
including a first region and a second region of differing 
resolutions; 

FIG. 3 illustrates the display screen of FIG. 2 
including a plurality of regions with differing resolutions; 

FIG. 4 illustrates another embodiment of a display 
system; 

FIG. 5 illustrates a plan view of another embodiment of 
a display screen including a first region and a second region 
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of differing resolutions; and 

FIG. 6 illustrates the display screen of FIG. 5 
including a plurality of regions with differing resolutions. 


Detailed Description of the Invention 

FIG. 1 illustrates a display system 10 including an eye 
tracking system 12, an eye-gaze tracking module 14, a display 
device 16, a display screen 18, and a scalable video decoder 
20. A user 22 gazes along an eye-gaze direction line 24 at 
the display screen 18 of the display device 16. The display 
device 16 may be included in any suitable system (e.g., 
computer monitor, television. Personal Digital Assistant, 
etc.). The eye tracking system 12 determines the eye-gaze 
direction line 24 by any suitable means (e.g., eye pupil 
movement, infrared, bright eye affect, etc.). The eye 
tracking system 12 sends the eye-gaze direction line 24 
information through a conduit 2 6 to the eye-gaze tracking 
module 14. The eye-gaze tracking module 14 extracts the eye- 
gaze direction line 24 information and determines an 
intersection point 28 where the eye-gaze direction line 24 
intersects with the display screen 18 (FIGS. 1, 2 and 3). 
The scalable video decoder 20 receives the intersection point 
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28 data from the eye-gaze tracking module 14 through the 
conduit 30. The scalable video decoder 20 receives an 
encoded video stream 32 from an encoded video source 34. The 
encoded video source 34 may be any suitable source (e.g., 
digital versatile disk, high definition TV broadcast, 
internet, tape recorder, computer system, etc.). The encoded 
video stream 32 is carried from the encoded video source 34 
through the conduit 3 6 to the scalable video decoder 20. The 
encoded video stream 32 may use any suitable video 
compression algorithm (e.g., MPEG-2, MPEG-4, H.263, etc.). 
MPEG-2 and MPEG-4 are compression standards of the Moving 
Picture Experts Group, and H.263 is a International 
Telecommunication Union compression standard. 

The scalable video decoder 20 generates a first set of 
"foveal" vision or higher resolution data 38 and a second set 
of "peripheral" vision or lower resolution data 40. The 
second set of lower resolution data 40 may optionally include 
data producing a lower brightness image than the first set of 
higher resolution data 38. The first set of higher 
resolution data 38 and the second set of lower resolution 
data 40 are sent through the conduit 42 to the display device 
16. 

FIG. 2 illustrates the display screen 18 of the display 
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device 16. The display screen 18 displays the first set of 
higher resolution data 38 in a first region 44 surrounding 
the intersection point 28. The intersection point 28 is the 
location where the user's 22 eye-gaze direction line 24 is 
intersecting with the display screen 18. The second set of 
lower resolution data 40 is displayed in a second region 4 6 
of the display screen 18. The second region 46 surrounds the 
first region 44 in the display screen 18. Additionally, the 
second region 46 of the display screen 18 may be dimmer than 
the first region 44 of the display screen 18. The signal 
bandwidth and the bits of data necessary to create the screen 
display 18 is reduced by having the first and second regions 
4 4 and 4 6 with different resolutions. For example, if the 
first and second regions 44, 4 6 where both at a high 
resolution of the 1024 x 780 pixels, the bandwidth and bits 
of data necessary would be greater than if the first region 
44 had a resolution equivalent to 1024 x 780 pixels but the 
second region had a resolution equivalent to 640 x 480 
pixels . 

FIG. 3 illustrates another embodiment of a display 
screen 18A with the first region 44 surrounded by the second 
region 46 including a plurality of regions 48, 50 and 52. 
The regions 48-52 have a resolution lower than the resolution 
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of the first region 44, 

FIG, 4 illustrates another embodiment of a display 
system lOA, The display system lOA includes the eye tracking 
system 12, the display device 16, the display screen 18B, the 
eye-gaze tracking module 14, a scalable video encoder 54, and 
a remote video decoder 56. A user 22A gazes along the eye- 
gaze direction line 24 at the display screen 18B of the 
display device 16. The eye tracking system 12 determines the 
eye-gaze direction line 24 by any suitable means (e.g., eye 
pupil movement;r infrared, bright eye affect, etc.). The eye 
tracking system 12 sends the eye-gaze direction line 24 
information through conduit 2 6 to the eye-gaze tracking 
module 14. The eye-gaze tracking module 14 extracts the eye- 
gaze direction line 24 information and determines the 
intersection point 28 where the eye-gaze direction line 24 
intersects with the display screen 18B (FIGS. 4, 5, and 6). 
The scalable video encoder 54 receives the intersection point 
28 data from the eye-gaze tracking module 14 through the 
conduit 30A. The scalable video encoder 54 receives a source 
video stream 60 from a video source 58. The video source may 
be any suitable source (e.g., camera, video cassette 
recorder, television, etc.). The source video stream 60 is 
carried from the video source 58 to the scalable video 

US010117 


encoder 54 through a conduit 62. 

The scalable video encoder 54 generates a first set of 
encoded higher resolution data 38A and a second set of 
encoded lower resolution data 40A. The first set of encoded 
higher resolution data 38A and the second set of encoded 
lower resolution data 4 OA may use any suitable video 
compression algorithm (e.g., MPEG-2, MPEG-4, H.263, etc.). 
The second set of encoded lower resolution data 40A may 
optionally include data producing a lower brightness image 
than the first set of encoded higher resolution data 38A. 
The first set of encoded higher resolution data 38A and the 
second set of encoded lower resolution data 40A are sent 
through a conduit 64 to the remote video decoder 56. The 
video decoder 56 decodes the first set of encoded higher 
resolution data 38A and the second set of encoded lower 
resolution data 40A and transmits the first set of higher 
resolution data 38 and the second set of lower resolution 
data 40 through conduit 66 to the display device 16. 

FIG. 5 illustrates the display screen 18B of the display 
device 16. The display screen 18 displays the first set of 
higher resolution data 40 in a first region 44A surrounding 
the intersection point 28. The intersection point 28 is the 
location where the user's 22A eye-gaze direction line 24 is 
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intersecting with the display screen 18B. The second set of 
lower resolution data 40 is displayed in a second region 4 5A 
of the display screen 18B. The bandwidth and bits of data 
necessary to create the screen display 18B is reduced by 
having the first region 44A exhibit a higher resolution than 
the second region 4 6A. 

FIG. 6 illustrates another embodiment of a display 
screen 18C with the first region 44A surrounded by the second 
region 46A including a plurality of regions 68, 70 and 72. 
The regions 68-72 have a resolution lower than the resolution 
of the first region 44A. Optionally, the regions 68-72 may 
be dimmer than the first region 44A. 

While embodiments of the present invention have been 
described herein for purposes of illustration, many 
modifications and changes will become apparent to those 
skilled in the art. For example, the display device 16 may 
be included in any suitable system (e.g., computer monitor, 
television. Personal Digital Assistant, etc.). Accordingly, 
the appended claims are intended to encompass all such 
modifications and changes as fall within the true spirit and 
scope of this invention. 
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